Self-Test 


A student evaluates the change in his academic progress 
after receiving a PDA — and then tests his peers. 


S everal years ago, upon enter- 
ing eighth grade, I was given 
a Microsoft iPaq Pocket PC as 
a birthday present. After using it for 
several months, I noticed a distinct 
improvement in my grades. During 
the rest of the school year, I noticed 
that the improved academic achieve- 
ment continued. I wondered if my ex- 
perience was a mere fluke or whether 
personal digital assistants (PDAs) in 
general — and Pocket PC devices in 
particular — could improve Millburn 
High School students’ academic 
achievement. I knew that the only 
way to confirm or refute this obser- 
vation was to conduct a controlled 
scientific experiment that would pro- 
duce empirical evidence. After four 
semester-long trials, my preliminary 
results show that Pocket PCs positive- 
ly affect academic achievement. 

My research attempted to answer 
three questions: Does the mere pos- 
session of the Pocket PC device in- 
crease academic achievement? Does 
Pocket PC usage increase academic 
achievement? Does software-specific 
usage of the Pocket PC devices in- 
crease academic achievement? 

First I obtained necessary approval 
from Millburn High School. I then 
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determined that the number of stu- 
dents currently possessing Pocket PCs 
was statistically insignificant. A sim- 
ple random sample determined which 
students would be offered the 12 
Pocket PC devices I acquired through 
the largesse of the Microsoft Corpora- 
tion. After all necessary parties signed 
consent forms, students were given 
Pocket PCs to use for a semester. Par- 
ticipants were asked to upload usage 
data several times during the semester 
and occasionally asked to comment 
about their Pocket PC experience. At 
the end of the semester the students 
returned the Pocket PCs. I received 
anonymous information about their 
academic performance from the 
school administration. 

The improvement in achievement 
was very evident. I looked to make 
sure there was no significant difference 
between the random group of students 
and the rest of the students in their 
grade level before they received Pocket 
PCs. Following this, I compared the 
achievement of each individual student 
to their previous achievement: rela- 
tive to previous performance, did they 
improve during the semester with a 
Pocket PC? Finally, I examined if there 
was a relative improvement compared 
to the rest of the grade level — that is, 
did the sample group improve more or 
less than the rest of the class? 

Before receiving Pocket PCs, the 
randomly selected group had similar 
grades to the rest of the students in 
their class. Afterwards, their perfor- 
mance increased 1 1 % compared to 
the entire grade level. When com- 
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pared to their previous 
performance as a group 
they increased 14%. 

This meant that a hy- 
pothetical Pocket PC 
user with a GPA of 2.5 
(corresponding to a C+) 
would increase to ap- 
proximately a 2.75 (or 
B-) by the end of the 
semester. 

Although my data about the ways 
in which students were using their 
devices was based on personal surveys 
and was far from comprehensive, 
it was intriguing to hear about and 
observe the many ways in which the 
devices were used. Many participants 
used their devices’ organizer capa- 
bilities, including Pocket Tasks and 
Pocket Calendar, to keep track of their 
assignments and projects in much the 
same way they would use a student 
organizer. Some participants took 
notes using Pocket Word and then 
e-mailed them to friends who were 
absent from class. Others entered data 
from their science labs into Pocket 
Excel for instant analysis. Only a few 
of the participants admitted to playing 
games or listening to music on their 
devices, although I suspect that the 
number was higher. 

The students as a whole had a 
positive experience with their Pocket 
PCs. Many were loathe to return the 
devices at the end of the semester and 
wanted to be included in subsequent 
trials. One student found her device 
too complicated and returned it after 
half a semester. 

This year I hope to conclude my 
project by conclusively answering the 
third question in my work — does 
specific application usage improve 
academic achievement? Over the 
summer I wrote two Pocket PC soft- 
ware applications that will help me 
answer this question. The first is an 
improved version of my usage track- 
ing software. The program will record 
what applications the student used 


and the amount of time 
spent using the applica- 
tion. The data will be 
submitted over the In- 
ternet anonymously to 
a server where it will be 
analyzed. The first ver- 
sion of the usage track- 
ing software had issues 
with data loss because 
the user would occasion- 
ally reset their device between data 
collections, erasing the data. 

The second application is a survey 
data collection application. When a 
student synchronizes his device with 
his computer, the “SurveyClient” pro- 
gram will connect to a server (using a 
Visual Studio. Net XML Web service) 
and download a new XML survey. 

The student can then take the survey 
when he or she has a free moment 
and does not have to be connected to 
the Internet. The next time the stu- 
dent synchronizes his or her device, 
the student’s response to the survey 
is submitted to the Web service. This 
allows fast and convenient data collec- 
tion that is easy to analyze. 

In my senior year, I plan to con- 
clude my study by deploying my new 
custom software and analyzing the 
data. I also hope to extend my project 
in size and scope by working with 
schools that have already distributed 
Pocket PC devices to their students. 

I hope to use my custom software to 
enable both the other schools and me 
to understand how Pocket PCs are 
being used in school. 



Alex Bick is a 16-year-old ju- 
nior attending Millburn High 
School, located in northern 
New Jersey. He has been help- 
ing students and teachers with 
technology since he joined the 
Computer Beer Leader pro- 
gram in seventh grade. He is being mentored by 
Dr. Chris Dede, a professor at Harvard Univer- 
sity. Bick will be presenting a Student Showcase 
of his work at NECC 2005 in Philadelphia, and 
an online version of his research can be found at 
http'.llwww. millburn. org! science !p dal. 
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